Machine learning for document authentication

ABSTRACT

Computer systems and methods are provided for using a machine learning system to analyze authentication information. First authentication information for a first transaction includes at least a first image that corresponds to a first identification document is received. First validation information that corresponds to a first validation fault is received from a validation system. Data storage of a machine learning system stores the first validation information. Second authentication information for a second transaction includes a second image that corresponds to a second image is received. The machine learning system determines a first validation value that corresponds to a probability that the second image includes the first validation fault. The first validation value is used to determine whether fault review criteria are met. In accordance with a determination that the fault review criteria are met, the second image is transmitted to the validation system.

RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.15/993,366 filed May 30, 2018, entitled “Machine Learning for DocumentAuthentication”, which is incorporated by reference herein in itsentirety.

TECHNICAL FIELD

This application relates generally to user authentication, and moreparticularly, to authentication using machine learning to generatealgorithms for analyzing image data.

BACKGROUND

Identification and verification of remote users is important for manynetwork communications and transactions. This is especially true withsensitive communications and important transactions when users areunacquainted and remote from each other. Traditionally, a personpresents a physical identification document for inspection by an agentwho compares a facial image of the identification document to thephysical appearance of the person. However, the conventional mechanismsof identifying and verifying users are inconvenient and inefficient, andresult in burdens for users.

SUMMARY

Accordingly, there is a need for systems and/or devices with moreefficient, accurate, and intuitive methods for analyzing an image forauthentication of a user. Such systems, devices, and methods optionallycomplement or replace conventional systems, devices, and methods foranalyzing an image for authentication of the user.

In some embodiments, a machine learning system is used to generate amodel that is used for analyzing image data. The image data includes,for example, an image of an identification document (e.g., anidentification document, such as a passport or other government-issueddocument, that includes an image of a user) and/or an image captured ofthe user (e.g., a recent “selfie” image of the user). A computing systemthat implements machine learning analyzes a set of training data thatinclude image data (e.g., image data included in a large number ofauthentication requests). The machine learning system is used togenerate a model that can be used for analysis of incomingauthentication requests. In some embodiments, a supervised trainingmodule of the machine learning system trains an authentication modelusing human tagged data (e.g., by iteratively classifying the data,comparing test data against human tags, and performing optimization tominimize classification error). In some embodiments, an unsupervisedtraining module of the machine learning system trains an authenticationmodel without using human input for classification (e.g., by encodingand clustering data to find potential groups and outliers). In someembodiments, the results of applying an authentication model (e.g.,trained by supervised learning and/or unsupervised learning) to a set ofauthentication request data is compared against the results of humanreview of the same set of authentication request data, and discrepanciesbetween the machine learning results and the human review results areused to alter the authentication model.

The disclosed subject matter includes, in one aspect, a computerizedmethod for receiving first authentication information for a firsttransaction. The first authentication information includes at least afirst image that corresponds to a first identification document. Themethod additionally includes receiving, from a validation system, firstvalidation information that corresponds to a first validation fault. Themethod additionally includes storing, by data storage of a machinelearning system, the first validation information. The methodadditionally includes receiving second authentication information for asecond transaction. The second authentication information includes asecond image that corresponds to a second identification document. Themethod additionally includes determining, using the machine learningsystem, a first validation value that corresponds to a probability thatthe second image includes the first validation fault. The methodadditionally includes determining, using the first validation value,whether fault review criteria are met. The method additionally includes,in accordance with a determination that the fault review criteria aremet, transmitting, to the validation system, the second image.

In some embodiments, a computer readable storage medium stores one ormore programs. The one or more programs comprise instructions, which,when executed, cause a device to receive first authenticationinformation for a first transaction. The first authenticationinformation includes at least a first image that corresponds to a firstidentification document. The one or more programs additionally compriseinstructions that cause the device to receive, from a validation system,first validation information that corresponds to a first validationfault. The one or more programs additionally comprise instructions thatcause the device to store, by data storage of a machine learning system,the first validation information. The one or more programs additionallycomprise instructions that cause the device to receive secondauthentication information for a second transaction. The secondauthentication information includes a second image that corresponds to asecond identification document. The one or more programs additionallycomprise instructions that cause the device to determine, using themachine learning system, a first validation value that corresponds to aprobability that the second image includes the first validation fault.The one or more programs additionally comprise instructions that causethe device to determine, using the first validation value, whether faultreview criteria are met. The one or more programs additionally compriseinstructions that cause the device to, in accordance with adetermination that the fault review criteria are met, transmit, to thevalidation system, the second image.

In some embodiments, a system includes one or more processors, memory,and one or more programs. The one or more programs are stored in thememory and configured to be executed by the one or more processors. Theone or more programs include instructions for receiving firstauthentication information for a first transaction. The firstauthentication information includes at least a first image thatcorresponds to a first identification document. The one or more programsadditionally include instructions for receiving, from a validationsystem, first validation information that corresponds to a firstvalidation fault. The one or more programs additionally includeinstructions for storing, by data storage of a machine learning system,the first validation information. The one or more programs additionallyinclude instructions for receiving second authentication information fora second transaction. The second authentication information includes asecond image that corresponds to a second identification document. Theone or more programs additionally include instructions for determining,using the machine learning system, a first validation value thatcorresponds to a probability that the second image includes the firstvalidation fault. The one or more programs additionally includeinstructions for determining, using the first validation value, whetherfault review criteria are met. The one or more programs additionallyinclude instructions for, in accordance with a determination that thefault review criteria are met, transmitting, to the validation system,the second image.

BRIEF DESCRIPTION OF THE DRAWINGS

So that the present disclosure can be understood in greater detail,features of various embodiments are illustrated in the appendeddrawings. The appended drawings, however, merely illustrate pertinentfeatures of the present disclosure and are therefore not limiting.

FIG. 1 is a system diagram of an authentication system and its context,in accordance with some embodiments.

FIG. 2 is a system diagram of a verification device, in accordance withsome embodiments.

FIG. 3A illustrates a reference image, in accordance with someembodiments.

FIG. 3B illustrates an authentication image, in accordance with someembodiments.

FIG. 4 illustrates an example of a reference image that includes afault, in accordance with some embodiments.

FIGS. 5-9 illustrate a validation user interface that receives faulttagging input, in accordance with some embodiments.

FIGS. 10-11 illustrate a validation user interface that is displayed toa user in accordance with a determination that a validation valuedetermined for a reference image meets review criteria, in accordancewith some embodiments.

FIG. 12 is a system diagram of a user device that captures a referenceimage and/or an authentication image, in accordance with someembodiments.

FIG. 13 illustrates a user device that displays a camera user interfacethat includes feedback regarding an image captured by a camera of theuser device.

FIGS. 14A-14C are a flow diagram illustrating a method for using machinelearning to generate an authentication model for analyzing image data,in accordance with some embodiments.

FIG. 15 is a flow diagram illustrating a method for using machinelearning to provide feedback at a device that captures an image for anauthentication request, in accordance with some embodiments.

FIG. 16 is a flow diagram illustrating a method for generatingadversarial tests to teach a machine learning system, in accordance withsome embodiments.

In accordance with common practice, some of the drawings may not depictall of the components of a given system, method, or device. Finally,like reference numerals denote like features throughout thespecification and figures.

DETAILED DESCRIPTION

Numerous details are described herein in order to provide a thoroughunderstanding of the example embodiments illustrated in the accompanyingdrawings. However, some embodiments may be practiced without many of thespecific details, and the scope of the claims is only limited by thosefeatures and aspects specifically recited in the claims. Furthermore,well-known processes, components, and materials have not been describedin exhaustive detail so as not to unnecessarily obscure pertinentaspects of the embodiments described herein.

The systems and methods described herein pertain to machine learningalgorithms for determining validity of one or more images thatcorrespond to an authentication request. These systems and methodsimprove on prior techniques for determining image validity of imagesfrom an authorization request. In some embodiments, an authenticationrequest is received from a requesting device (e.g., in connection with apayment or other secure transaction). In some embodiments, theauthentication request includes an image of an identification document,such as a passport (e.g., that is associated with a user for whom asecure transaction is being performed). In some embodiments, theauthentication request includes an image of the user (e.g., a recent“selfie” image). In response to the authentication request, anauthentication system determines validity of the image of theidentification document and/or compares the image of the user with theimage of the identification document to determine whether matchingcriteria are met. Information about previously analyzed authenticationrequests is used by the authentication system to generate a model thatis used to evaluate incoming authentication requests. In someembodiments, the model is generated via supervised machine learning(e.g., the model is generated and/or refined using validation faultslabeled by a human reviewer). In some embodiments, the model isgenerated via unsupervised machine learning (e.g., the model isgenerated based on an unlabeled set of data). The model is used todetermine whether criteria are met for transmitting an image from anauthentication request to a validation system.

In some embodiments, the authentication systems described hereindecrease the time required for human review of identification documents(e.g., by using a model generated by a machine learning system toanalyze an image and provide information to human reviewers aboutinformation generated by the analysis) and/or reduce the extent of humanreview used for authenticating identification documents (e.g., by usingthe model to determine whether to bypass human review). Using machinelearning as described herein to reduce the extent of human review and/orto reduce the time required for human review improves the authenticationdevice by making the processing of authentication requests faster andmore efficient, with less required human interaction, which in turnreduces the processing and power used by an authentication server and/ora validation device.

Additionally, there is a need to analyze identification documents todetermine whether the identification documents are authentic, whilemaintaining compliance of the data set with applicable securitystandards. For example, the security of authentication data ismaintained in compliance with the payment card industry data securitystandard (PCI DSS). Whereas existing machine learning systems may relyon labels assigned to a public set of data, PCI DSS compliance requireslimited access to validation data sets. However, the usability of amodel generated by a machine learning system improves as the size of theavailable data set increases. Various embodiments described hereinadvantageously perform machine learning using data generated by alimited set of human validation reviewers. Using a model generated by amachine learning system to perform image analysis reduces the amount oftime required for human review of authorization requests, whichincreases the rate of authorization request processing while maintainingsecurity of processed data.

FIG. 1 is a system diagram of an authentication server 100 (alsoreferred to herein as a “machine learning system”), in accordance withsome embodiments. The authentication server 100 typically includes amemory 102, one or more processor(s) 104, a power supply 106, aninput/output (I/O) subsystem 108, and a communication bus 110 forinterconnecting these components.

The processor(s) 104 execute modules, programs, and/or instructionsstored in the memory 102 and thereby perform processing operations.

In some embodiments, the memory 102 stores one or more programs (e.g.,sets of instructions) and/or data structures, collectively referred toas “modules” herein. In some embodiments, the memory 102, or thenon-transitory computer readable storage medium of the memory 102 storesthe following programs, modules, and data structures, or a subset orsuperset thereof:

-   -   an operating system 120;    -   an authentication request data module 122, which stores        information for a plurality of authentication requests 124        (e.g., a first authentication request 124 a, a second        authentication request 124 b . . . Nth authentication request        124N), each of which includes information such as identification        information 126 (e.g., unique identification, user name, user        password, user residential information, user phone number, user        date of birth, and/or user e-mail), a reference image 300 a,        and/or an authentication image 350 a; and    -   a machine learning module 126 that uses supervised training        module 130, unsupervised training module 132, and/or adversarial        training module 134 to generate authentication model 136 (e.g.,        by analyzing validation information 128 corresponding to        multiple authentication requests 124).

The above identified modules (e.g., data structures and/or programsincluding sets of instructions) need not be implemented as separatesoftware programs, procedures, or modules, and thus various subsets ofthese modules may be combined or otherwise re-arranged in variousembodiments. In some embodiments, the memory 102 stores a subset of themodules identified above. In some embodiments, a remote authenticationdatabase 152 and/or a local authentication database 142 store a portionor all of one or more modules identified above (such as authenticationrequest data 122, validation information 128, and/or authenticationmodel 136). Furthermore, the memory 102 may store additional modules notdescribed above. In some embodiments, the modules stored in the memory102, or a non-transitory computer readable storage medium of the memory102, provide instructions for implementing respective operations in themethods described below. In some embodiments, some or all of thesemodules may be implemented with specialized hardware circuits thatsubsume part or all of the module functionality. One or more of theabove identified elements may be executed by one or more of theprocessor(s) 104. In some embodiments, one or more of the modulesdescribed with regard to the memory 102 is implemented in the memory 202of a validation device 201 (FIG. 2) and executed by the processor(s) 204of the validation device 201. In some embodiments, one or more of themodules described with regard to the memory 102 is implemented in thememory 1202 of a user device 156 (FIG. 12) and executed by theprocessor(s) 1204 of the user device 156. For example, machine learningmodule 126 is distributed across authentication server 100, validationdevice(s) 201, validation server 200, and/or user device 156

In some embodiments, for each received authentication request 124,machine learning module 126 stores data for training authenticationmodel, such as:

-   -   Channel via which the authentication request 124 was received        (e.g., mobile, web, and/or application programming interface        (API))    -   internet protocol (IP) address from which the authentication        request 124 was received    -   camera information (e.g., camera model, device of which the        camera is a component, and/or orientation of camera relative to        device);    -   transaction origination location (e.g., country, state,        province, and/or city);    -   type of identification document captured in reference image 300        (e.g., passport, driver license, and/or corporate identification        badge);    -   identification document origination location (e.g., country,        state, province, and/or city); and/or    -   validation information 128 (e.g., including a verification        outcome such as verified, fraud detected, and/or rejected).

In some embodiments, generating the authentication model 136 includesgenerating a regression algorithm for prediction of continuous variables(e.g., perspective transformation of a reference image 300 and/or a morecomplex transformation describing bending of passport pages).

In some embodiments, the I/O subsystem 108 communicatively couples thecomputing system 100 to one or more devices, such as a localauthentication database 142, a remote authentication database 152, arequesting device 154, a user device 156, validation server 200 (e.g.,including one or more server components (e.g., one or more processor(s)and memory) and/or modules described with regard to authenticationserver 100) and/or validation device(s) 201 via a communications network150 and/or via a wired and/or wireless connection. In some embodiments,the communications network 150 is the Internet.

The communication bus 110 optionally includes circuitry (sometimescalled a chipset) that interconnects and controls communications betweensystem components.

Typically, an authentication system for processing authenticationrequests includes a server computer system 100 that is communicativelyconnected to one or more validation devices 201 (e.g., via a network 150and/or an I/O subsystem 108). In some embodiments, the authenticationsystem receives an authentication request (e.g., from a user device 156that captures an image of a user or from a requesting device 154 thatreceives an image from user device 156). For example, the authenticationrequest is a request to authenticate the identity of a user 124 (e.g., auser that is a party to a transaction or a user that is requestingaccess to a system or physical location). Requesting device 154 is, forexample, a device of a merchant, bank, transaction processor, computingsystem or platform, physical access system, or another user. User device156 is described in further detail with regard to FIG. 12 below.

In some embodiments, an authentication request includes a referenceimage 300 (e.g., 300 a, as illustrated in FIG. 3A, 300 b, as illustratedin FIG. 4, 300 c as illustrated in FIG. 10, or 300 d, as illustrated inFIG. 13). For example, reference image 300 is an image of anidentification document for a user 124). In some embodiments, anauthentication request includes an authentication image 350 (e.g., 350a, as illustrated in FIG. 3B). For example, authentication image 350 isan image, series of images, and/or video of the user 124 captured by auser device 156, such as a recent “selfie” of the user 124. In someembodiments, an authentication request includes an authentication image350 and the authentication system locates a reference image 300 thatcorresponds to the user 124 that provided the authentication image 350.

In some embodiments, the authentication server 100 causes a validationdevice 201 to display all or a part of a reference image 300 and/or allor a part of an authentication image 350 for human review. In someembodiments, the validation device 201 receives input that correspondsto a determination of whether authentication is successful (e.g., basedon whether reference image 300 is sufficiently similar to theauthentication image 350). In some embodiments, validation device 201transmits validation information 128 that corresponds to a determinationof whether authentication is successful (e.g., indicating that areference image 300 is a match for authentication image 350, indicatingthat reference image 300 is not a match for authentication image 350,and/or indicating that a fault was detected in reference image 300and/or authentication image 250).

FIG. 2 is a system diagram of a validation device 201 that displaysvalidation information and/or receives validation input, in accordancewith some embodiments. The validation device 201 typically includes amemory 202, one or more processor(s) 204, a power supply 206, aninput/output (I/O) subsystem 208, and a communication bus 210 forinterconnecting these components. In some embodiments, validation device201 is coupled to a validation server 200 (e.g., that receives data fromauthentication server 100, transmits data to authentication server 100,and/or manages validation device(s) 201 (e.g., by transmitting data tovalidation device(s) 201 and/or receiving data from validation device(s)201)). In some embodiments, validation device(s) 201 are managed byauthentication server 100 (e.g., including transmitting data toauthentication server 100 and/or receiving data from authenticationserver 100).

The processor(s) 204 execute modules, programs, and/or instructionsstored in the memory 102 and thereby perform processing operations.

In some embodiments, the memory 202 stores one or more programs (e.g.,sets of instructions) and/or data structures, collectively referred toas “modules” herein. In some embodiments, the memory 202, or thenon-transitory computer readable storage medium of the memory 202 storesthe following programs, modules, and data structures, or a subset orsuperset thereof:

-   -   an operating system 220; and    -   a validation user interface 222, which displays information        (e.g., identification information 126 b, a reference image 300 b        and/or an authentication image 350 b) and/or includes one or        more input controls for receiving validation input.

The above identified modules (e.g., data structures, and/or programsincluding sets of instructions) need not be implemented as separatesoftware programs, procedures, or modules, and thus various subsets ofthese modules may be combined or otherwise re-arranged in variousembodiments. Furthermore, the memory 202 may store additional modulesnot described above. In some embodiments, the modules stored in thememory 202, or a non-transitory computer readable storage medium of thememory 202, provide instructions for implementing respective operationsin the methods described below. In some embodiments, some or all ofthese modules may be implemented with specialized hardware circuits thatsubsume part or all of the module functionality. One or more of theabove identified elements may be executed by one or more of theprocessor(s) 204. In some embodiments, one or more of the modulesdescribed with regard to the memory 202 is implemented in the memory 102of a server computing system 100 (FIG. 1) and executed by theprocessor(s) 104 of the server computing system 100.

In some embodiments, the I/O subsystem 208 communicatively couples thevalidation device 201 to one or more devices (e.g., user input device212, output device 214, and/or server computer system 100) via acommunications network 150 and/or via a wired and/or wirelessconnection. In some embodiments, a user input device 212 and/or anoutput device 214 are integrated with validation device 201 (e.g., as atouchscreen display). In some embodiments, user input device 212 and/oroutput device 214 are peripheral devices communicatively connected tovalidation device 201. In some embodiments, a user input device 212includes a keyboard and/or a pointer device such as a mouse, touchpad,and/or stylus. In some embodiments, output device 214 includes, forexample, a display and/or a speaker.

In some embodiments, validation user interface 222, user input device212, and/or output device 214 are incorporated into server computersystem 100 (e.g., rather than being implemented in a separate validationdevice 201).

The communication bus 210 optionally includes circuitry (sometimescalled a chipset) that interconnects and controls communications betweensystem components.

In some embodiments, validation device 201 (and/or authentication server100) displays, by a validation user interface 222 displayed by outputdevice 214, identification information 126, reference image 300, and/orauthentication image 350. In some embodiments, a reviewer viewingvalidation user interface 222 provides input to validation device viauser input device 212. The input provided by the reviewer includes, forexample, information regarding the validity of reference image 300(e.g., the results of one or more validity checks for reference image300), information regarding the validity of authentication image 350(e.g., the results of one or more validity checks for authenticationimage 350), and/or information regarding the similarity of referenceimage 300 and authentication image 350 (e.g., the results of one or moresimilarity checks for reference image 300 and authentication image 350).

FIG. 3A illustrates a reference image 300 a, in accordance with someembodiments. Reference image 300 a is, for example, an image of anidentification document 302 that includes a facial image 304 of a user.For example, reference image 300 a is an image of an identificationcard, a driver's license, a passport, a financial instrument (e.g.,credit card or debit card), or a facility access card.

FIG. 3B illustrates an authentication image 350 a that includes a facialimage 352 of the user, in accordance with some embodiments. For example,authentication image 350 a is a “selfie” captured using user device 156.Authentication image 350 a is, e.g., a still image or an image extractedfrom a video. In some embodiments, the authentication image 350 aincludes a series of images or a video (e.g., used for determining thatthe authentication image 350 meets liveness requirements).

FIG. 4 illustrates an example of a reference image 300 b that has beencaptured of the reference document 302 after the reference document 302has been punched (as indicated by the puncture hole 404 that is visiblein reference image 300 b).

FIGS. 5-9 illustrate a validation user interface 222 that receives faulttagging input, in accordance with some embodiments. In some embodiments,validation user interface 222 is displayed by output device 214 to ahuman reviewer (e.g., to facilitate review of authentication informationfor an authentication request).

In some embodiments, validation user interface 222 includes referenceimage 300 b and/or authentication image 350 b (e.g., to allow a reviewerto compare the images and determine whether the images match). Forexample, in FIG. 5, validation user interface 222 includes referenceimage 300 b, authentication image 350 b, and a match control 502 (e.g.,receive reviewer input indicating whether or not reference image 300 bin a match for authentication image 350 b). In some embodiments,validation user interface 222 includes a progression control 508 (e.g.,for advancing to a subsequent set of authentication review data, asubsequent interface and/or a subsequent stage of validation userinterface 222). For example, in a case where no image fault is presentin reference image 300 b or authentication image 350 b, a human reviewermay use match control 502 to indicate whether or not authenticationimage 350 b is a match for reference image 300 b and then useprogression control 508 to advance to a subsequent authenticationreview.

In some embodiments, validation user interface 222 includes one or morecontrols for indicating that the human reviewer has detected a faultand/or for identifying a fault. For example, in FIG. 5, validation userinterface 222 includes an image fault detection control 504 (e.g., aselectable button) that, when activated, causes validation device 201 tostore and/or transmit data indicating that a fault has been detected. Insome embodiments, activation of fault detection control 504 initiates afault location indication mode. For example, in FIG. 5, becausereference image 300 b includes an indication of a puncture hole 404, thehuman reviewer has provided input using a user input device 212 (e.g.,as indicated by pointer 506) at a location that corresponds to imagefault detection control 504.

In FIG. 6, in response to detection of the user input at the locationthat corresponds to image fault detection control 504, a fault locationindication stage of validation user interface 222 has been displayed. Insome embodiments, validation user interface 222 (e.g., image faultdetection control 504) and/or pointer 506 is visually altered toindicate that fault location indication mode is active. In someembodiments, validation user interface 222 displays instructions toselect the fault location, as indicated at 602. User input device 212has been used to select a region that corresponds to punch mark 404, asindicated by the selection box 604. In some embodiments, multiple faultregions may be selected in reference image 300 b and/or inauthentication image 350 b.

In FIG. 7, while the region that corresponds to the image fault (e.g.,punch 404) is selected, input by user input device 212 is detected at alocation that corresponds to progression control 508 (e.g., whilepointer 506 is at the location that corresponds to progression control508).

In FIG. 8, in response to detection of the input by user input device212 at the location that corresponds to progression control 508, a faulttype identification stage of validation user interface 222 has beendisplayed. In some embodiments, in the fault type identification stageof validation user interface 222, a plurality of fault types (e.g., asdiscussed further below with regard to operation 1206) are displayed asselectable options. In some embodiments, validation user interface 222displays instructions to select the fault type, as indicated at 802. InFIG. 8, an input by the human reviewer has selected the “puncheddocument” fault type, as indicated by the “selected” state of toggle804. In some embodiments, multiple fault types may be assigned to afault identified in a selected fault region (e.g., as indicated byselection box 604).

In FIG. 9, while the fault type that corresponds to the image fault isselected, input by user input device 212 is detected at a location thatcorresponds to progression control 508 (e.g., while pointer 506 is. atthe location that corresponds to progression control 508). In someembodiments, in response to detection of the input at the location thatcorresponds to progression control 508, validation information 128 istransmitted from validation device 201 to authentication server 100(e.g., for use by machine learning module 126). For example, thevalidation information 128 includes the image 300 b, informationindicating the fault type identified as described with regard to FIG. 8,and/or information indicating the region selected by selection box 604.

It will be recognized that the description of validation user interface222 provided with reference to FIGS. 5-9 is one illustrative example ofan approach to a user interface for image fault tagging, and thatvarious controls, inputs, and/or sequences of operations are used inaccordance with various embodiments for human tagging of image faults.For example, in some embodiments, an indication that an image fault hasbeen detected may be provided by selection of at least a portion ofreference image 300 b or at least a portion of authentication image 350b (e.g., without display and/or activation of a control 504 forindicating detection of an image fault).

In some embodiments, the machine learning module 126 includes asupervised training module 130 that uses image fault data generated byhuman reviewers (e.g., as described with regard to FIG. 5-9) to trainauthentication model 136 to identify image faults. For example, themachine learning module 126 uses the human-tagged image fault data togenerate rules of an authentication model 136 for determining thepresence and/or the location of faults in images included in anauthentication request 124. In some embodiments, the machine learningmodule 126 applies the generated authentication model 136 to performimage analysis (e.g., using computer vision techniques) for identifyingimage faults in images (such as a reference image 300 and/or anauthentication image 350) of a received authentication request 124. Insome embodiments, in accordance with a determination by machine learningmodule 126 that a fault is present in an image, the image that includesthe fault is displayed to a human reviewer for further review (e.g.,with a prompt to identify a location of fault). In some embodiments, inaccordance with a determination by machine learning module 126 that nofault is present in an image, the image that includes the fault isdisplayed to a human reviewer for further review (e.g., with a prompt toidentify a location of fault). In some embodiments, the human input isfed back into the supervised machine learning system.

In some embodiments, the machine learning module 126 analyzes a set ofincoming authentication requests 124 (e.g., authentication requestsreceived during a predetermined window of time). In some embodiments,machine learning module 126 includes an unsupervised training module 132that trains authentication model 136 using the set of incomingauthentication requests without using human-tagged validationinformation (e.g., in addition to or in lieu of a supervised trainingmodule 130 that trains authentication model 136 using human-taggedvalidation information (e.g., as described with regard to FIGS. 5-9)).

In some embodiments, the machine learning module 126 generates multipleauthentication models 136. For example, a respective authenticationmodel 136 is customized based on one or more parameters (e.g., asdefined for an entity (e.g., bank, merchant, and/or class of users),type of device, type of identification, and/or type of transaction).

In some embodiments, the machine learning module 126 determines a subsetof the set of incoming authentication requests 124 that correspond toinvalid authentication request activity (e.g., authentication requeststhat are reported for fraud or that are determined to include a fault).In some embodiments, the unsupervised training module 132 determinesthat, in the subset of the set of incoming authentication requests 124,a number of invalid authentication requests exceeds a threshold numberof invalid submissions and/or a rate of fraudulent activity exceeds athreshold level.

In some embodiments, a validation fault (e.g., the second validationfault described with regard to operation 1430, described with regard toFIG. 14 below) is an invalidity characteristic (e.g., an indication ofan internet protocol (IP) address, geographical area, device type (e.g.,a model that is no longer available), camera type, document type,transaction type, time at which request is received, and/or verificationoutcome) of the subset of the set of incoming authentication requests.In some embodiments, weights are applied to the first validation fault,the second validation fault, and/or any additional validation faults. Insome embodiments, weights are adjusted over time (e.g., in response toregional activity, transaction types determined to be problematic,etc.).

FIGS. 10-11 illustrate a validation user interface 222 that is displayedto a user in accordance with a determination that a validation valuedetermined for reference image 300 c meets review criteria, inaccordance with some embodiments. For example, machine learning module126 has determined that a fault is present in reference document 1002captured in image 300 c (e.g., reference document 302 has been punched,as indicated by punch mark 1004). In some embodiments, validation userinterface 222 is displayed by output device 214 to a human reviewer. Forexample, in FIG. 10 the validation user interface 222 is displayed tothe human reviewer with an image fault detected label 1012 and a taskprompt 1010 (e.g., prompting the reviewer to confirm presence of thefault and/or to identify a location of the fault).

In FIG. 11, user input device 212 has been used (e.g., as indicated bypointer 1008) to select a region (e.g., as indicated by the selectionbox 1006) that corresponds to the image fault (e.g., punch mark 1004).

In some embodiments, a machine learning system is used to analyze imagescaptured by a user device 156 to provide feedback regarding the capturedimage at the mobile device. For example, the feedback is provided priorto generation of an authorization request.

FIG. 12 is a system diagram of user device 156 (e.g., a device thatincludes and/or is connected to a camera, such as a personal computer ormobile device) that captures a reference image 300 d and/or anauthentication image 350 d, in accordance with some embodiments. Theuser device 156 typically includes a memory 1202, one or moreprocessor(s) 1204, a power supply 1206, an input/output (I/O) subsystem1208, and a communication bus 1210 for interconnecting these components.

The processor(s) 1204 execute modules, programs, and/or instructionsstored in the memory 1202 and thereby perform processing operations.

In some embodiments, the memory 1202 stores one or more programs (e.g.,sets of instructions) and/or data structures, collectively referred toas “modules” herein. In some embodiments, the memory 1202, or thenon-transitory computer readable storage medium of the memory 1202,stores the following programs, modules, and data structures, or a subsetor superset thereof:

-   -   an operating system 1220;    -   a camera operation module 1224 that controls camera functioning        (e.g., image capture by camera 1214 and and/or one or more        camera parameters of camera 1214);    -   image storage module 1226 that stores images captured by camera        1214 (e.g., captured reference image 300 d and/or a captured        authentication image 350 d);    -   a machine learning module 1228 that includes an error correction        model 1230 for analyzing captured images to determine presence        of image errors; and    -   a camera user interface 1232 that displays captured images        (e.g., captured reference image 300 d and/or a captured        authentication image 350 d) and/or displays feedback to a user        regarding the captured images.

The above identified modules (e.g., data structures, and/or programsincluding sets of instructions) need not be implemented as separatesoftware programs, procedures, or modules, and thus various subsets ofthese modules may be combined or otherwise re-arranged in variousembodiments. Furthermore, the memory 1202 may store additional modulesnot described above. In some embodiments, the modules stored in thememory 1202, or a non-transitory computer readable storage medium of thememory 1202, provide instructions for implementing respective operationsin the methods described below. In some embodiments, some or all ofthese modules may be implemented with specialized hardware circuits thatsubsume part or all of the module functionality. One or more of theabove identified elements may be executed by one or more of theprocessor(s) 1204.

In some embodiments, one or more of the modules described with regard tothe memory 1202 (for example, the machine learning module 1228) isimplemented in the memory 102 of authentication server 100 (FIG. 1) andexecuted by the processor(s) 104 of the server computing system 100. Forexample, authentication server 100 uses machine learning module 1228and/or machine learning module 126 to analyze a plurality ofauthorization requests 124 and/or to generate error correction module1230. In some embodiments, error correction model 1230 and/or updates toerror correction model 1230 are transmitted from authentication server100 to user device 156. In some embodiments, images captured by userdevice 156, including images that include errors, are transmitted byuser device 156 to authentication server 100 (e.g., for training errorcorrection module 1230 and/or included in an authentication request). Insome embodiments, images captured by user device 156 are used by machinelearning module 1228 executing on user device 156 to train errorcorrection model 1230. In some embodiments, error correction model 1230is applied by user device 156 to an image captured by user device 156(e.g., to provide feedback to a user regarding the quality of thecaptured image and/or any faults detected in the captured image). Insome embodiments, error correction model 1230 is applied to a set ofimages, a human reviewer reviews the same set of images (e.g., usingvalidation device 200), and error correction model 1230 is updated basedon inconsistencies between the human review results and error correctionmodel results (e.g., difference in confidence level exceeding athreshold inconsistency value).

In some embodiments, the I/O subsystem 1208 communicatively couples thevalidation device 1202 to one or more devices (e.g., user input device1212, camera 1214, and/or output device 1216) via a communicationsnetwork 150 and/or via a wired and/or wireless connection. In someembodiments, a user input device 1212 and/or an output device 1216 areintegrated with user device 156 (e.g., a touchscreen display). In someembodiments, user input device 1212 and/or output device 1216 areperipheral devices communicatively connected to user device 156. In someembodiments, a user input device 1212 includes a keyboard and/or apointer device such as a mouse, touchpad, and/or stylus. In someembodiments, user input device 1212 includes a microphone. In someembodiments, output device 1216 includes a display and/or a speaker.

The communication bus 1210 optionally includes circuitry (sometimescalled a chipset) that interconnects and controls communications betweensystem components.

FIG. 13 illustrates a user device 156 that displays a camera userinterface 1232 that includes feedback (e.g., a feedback message 1304)regarding an image (e.g., reference image 300 d) captured by a camera ofuser device 156. For example, the displayed user interface 1302 includesa reference image 300 d in which reference document 1306 is partiallycovered, as indicated by the blank space at 1308. In some embodiments,user device 156 applies an error correction model 1230 to analyze acaptured image (e.g., after capturing the image). In some embodiments,user device 156 transmits a captured image to authentication server 100,authentication server 100 uses error correction model 1230 to analyzethe captured image, and authentication server 100 transmits image errorinformation to user device 156.

In some embodiments, in accordance with a determination that one or moreerrors are identified in a captured image (e.g., one or more errorsindicated in image error information received from authentication server100 and/or one or more errors determined by user device 156 using errorcorrection model 1230), camera operation module 1224 adjusts one or moreparameters of camera 1214 that correspond to at least one of the one ormore errors. For example, in accordance with a determination that aglare error is identified in a captured image, camera operation module1224 disables the flash. In some embodiments, camera operation module1224 adjusts one or more camera parameters and (e.g., subsequently)automatically captures a new image.

In some embodiments, in accordance with a determination that one or moreerrors are identified in a captured image, camera user interface 1232displays a control (e.g., image re-capture button 1310) for capturing anew image. For example, user device 156 displays image re-capture button1310 that, when activated (e.g., by a contact on a touchscreen displayof user device 156 that corresponds to a location of image re-capturebutton), causes a new image to be captured by the camera of user device156 and/or that causes a camera operation user interface (not shown) tobe displayed.

FIGS. 14A-14C are a flow diagram illustrating a method 1400 for usingmachine learning to generate an authentication model 136 for analyzingimage data, in accordance with some embodiments. The method is performedat an authentication server 100, a validation server 200, user device156, and/or a validation device 201. For example, instructions forperforming the method are stored in the memory 102 and executed by theprocessor(s) 104 of the authentication server computer system 100. Insome embodiments, part or all of the instructions for performing themethod are stored in the memory 202 and executed by the processor(s) 204of the validation device(s) 201. In FIGS. 14A-14C, dotted lines are usedto indicate optional operations.

The device receives (1402) first authentication information (e.g.,authentication request 124 a) for a first transaction. The firstauthentication information includes at least a first image (e.g., astill image, a video, and/or multiple frames from a video) thatcorresponds to a first identification document 302. For example, thefirst image is a reference image 300, such reference image 300 bdescribed with regard to FIG. 4. In some embodiments, the authenticationrequest 124 is received from requesting device 154 or user device 156.

In some embodiments, the device (1404) causes the first image to bedisplayed by a display (e.g., output device 214) of the validationsystem (e.g., validation server 200 and/or validation device 201). Forexample, the device transmits the first image to the validation device201 and/or transmits instructions to display the first image.

The device receives (1406) from the validation system (e.g., validationserver 200 and/or validation device 201), first validation information128 (e.g., information indicating a type of fault and/or a location ofthe fault) that corresponds to a first validation fault (e.g., a punchmark 404 as described with regard to FIGS. 4-9). In some embodiments,the first validation information indicates a location in the first image(e.g., reference image 300 b) that corresponds to the first validationfault (e.g., punch mark 404). In some embodiments, the validationinformation 128 includes the first image.

In some embodiments, validation faults include fraud indications,including, for example:

-   -   an indication that authentication image 350 is a digital copy of        reference image 300;    -   an indication that image manipulation occurred in the received        reference image 300 and/or authentication image 350;    -   an indication that reference image 300 has been cloned (e.g. at        least a (modified) portion of reference image 300 has been used        to create a submitted authentication image 350) and/or an        indication that authentication image 350 has been cloned (e.g.,        at least a (modified) portion of authentication image 350 has        been used to create a submitted reference image 300);    -   an indication that an image submitted as reference image 300        and/or authentication image 350 has been spliced (e.g., the        image is generated from at least a portion of an image that is        distinct from reference image 300 and authentication image 350);    -   an indication that text has been inserted into (and/or is        missing from) reference image 300 and/or authentication image        350;    -   an indication that the reference document depicted in reference        image 350 is a punched document (e.g., a passport or license        that has been punched to indicate that the document is no longer        valid);    -   an indication that data-masking (blurring, masking, and/or        pixelization) has been applied to an image (e.g., reference        image 300 and/or authentication image 350);    -   an indication that an irregularity is detected in a feature of        reference image 300 that corresponds to a feature of the        document type of reference document 302;    -   an indication that one or more text features (e.g., font type,        font size, inclination, and/or kerning) that corresponds to the        reference document 302 is different from a font type detected in        reference image 300; and/or    -   an indication that a layout irregularity is detected in        reference image 300 (e.g., variation from an average layout        determined for a document type of reference document 302 and/or        an incorrect location of data and/or graphical elements of        reference document 302).

In some embodiments, validation faults include readability faults,including, for example:

-   -   a document finding error (e.g., no document is found in        reference image 300 and/or only a portion of a document is found        in reference image 300);    -   an optical character recognition (OCR) fault (e.g., text in        image not recognized by OCR and/or no text detected in reference        image 300 of an identification document of a type that requires        text);    -   blurriness (e.g., document does not meet sharpness and/or        contrast criteria);    -   motion blur (e.g., smearing or other indication that document        moved during image and/or video capture);    -   glare (e.g., a portion of an image and/or text in the image is        obscured by glare), and/or    -   unintentional data-masking (e.g., fingers or other objects are        visible in the submitted image, a portion of the reference        document 302 is covered, and/or an insufficient portion of the        reference document 302 or face 352 is visible in the image).

The device stores (1408), by data storage of a machine learning system(e.g., memory 102 of authentication server 100 that includes machinelearning module 126), the first validation information 128. In someembodiments, the stored first validation information is used by machinelearning module 126 (e.g., using supervised training module 130) totrain authentication model 136. In some embodiments, the machinelearning module 126 uses validation information 128 that corresponds toa plurality (typically a very large number) of authentication requests124 to generate authentication model 136. For example, data points frommillions of transactions are used to train the authentication model 136.

The device receives (1410) second authentication information (e.g.,authentication request 124 b) for a second transaction. The secondauthentication information includes a second image (e.g., as shown at300 c in FIG. 10) that corresponds to a second identification document(e.g., distinct from the first identification document).

In some embodiments, the machine learning module 126, having trainedauthentication module 126 to recognize the punched document fault basedon input received via validation user interface 222 (e.g., as describedwith regard to FIGS. 5-9), uses the trained authentication model 136 todetermine whether the same fault is present in an image included in anincoming authentication request (e.g., as described with regard to FIGS.10-11).

The device determines (1412), using the machine learning system, a firstvalidation value that corresponds to a probability that the second imageincludes the first validation fault. In some embodiments, the machinelearning module 126 applies authentication model 136 trained usingsupervised training 130. For example, the machine learning module 126applies authentication model 136, which has been trained using humantagged fault information (e.g., fault information indicating the punchmark 404 in reference image 300 b) included in validation information128, to determine a confidence value indicating a probability that thevalidation fault (e.g., punch mark 1004) is present in the second image(e.g., reference image 300 c). In some embodiments, the machine learningmodule 126 applies authentication model 136 trained using unsupervisedtraining 132. Typically, validation information (e.g., indicatingpresence and/or location of punch marks) indicating informationregarding faults detected in image from a large number of authenticationrequests is used to train the authentication model 136 (e.g., prior tousing authentication model 136 to identify a fault in reference imagessuch as reference image 300 c). In some embodiments, the devicedetermines (e.g., simultaneously or substantially simultaneously), usingthe machine learning system, validation values for a set of validationfaults that includes the first validation fault and at least oneadditional validation fault (e.g., the second validation fault asdescribed with regard to 1430).

The device determines (1414), using the first validation value, whetherfault review criteria are met. For example, the fault review criteriainclude a criterion that is met in accordance with a determination thatthe first validation value exceeds a threshold uncertainty value. Inaccordance with a determination that the fault review criteria are met(e.g., indicating that human review is required), the method proceeds tooperation 1420. In some embodiments, in accordance with a determinationthat the fault review criteria are not met, the method proceeds tooperation 1416. In some embodiments, the device determines whether faultreview criteria are met for each respective validation fault of a set ofvalidation faults that includes the first validation fault and at leastone additional validation fault.

In some embodiments, in accordance with a determination that the faultreview criteria are not met, the second image is displayed (1416)without information indicating that a fault is detected. For example,authentication device 100 transmits information to validation device 201(e.g., information indicating that fault review criteria are not metand/or that no fault is detected) that causes validation device 201 todisplay a validation user interface 222 as described with regard to FIG.5. In some embodiments, validation user interface 222 (e.g., asdescribed with regard to FIG. 5) is displayed without image faultdetection control 504 (e.g., because no fault is detected). In someembodiments, validation user interface 222 (e.g., as described withregard to FIG. 5) is displayed with image fault detection control 504(e.g., to allow the human reviewer to override the determination by themachine learning module 126).

In some embodiments, in accordance with a determination that the faultreview criteria are not met, the device (1418) transmits authenticationinformation to a remote device. For example, authentication server 100transmits authentication information to requesting device 154 and/oruser device 156. In some embodiments, the authentication informationincludes information indicating that no faults are detected in theimage. In some embodiments, the authentication information includesinformation indicating a validation value corresponding to a confidencelevel that no faults are detected in the image. In some embodiments, theauthentication information includes information indicating that anauthentication request is approved (e.g., in accordance with adetermination that no fault is detected).

In some embodiments, in accordance with a determination that the faultreview criteria are met, the device transmits (1420), to the validationsystem (e.g., for display by the first display or at a second displaythat is distinct from the first display), the second image. In someembodiments, authentication device 100 transmits information indicatingthat fault review criteria are met, that one or more faults aredetected, and/or determined fault types of one or more detected faults.In some embodiments, the information transmitted by authenticationdevice 100 to the validation system causes validation device 201 todisplay a validation user interface 222 as described with regard toFIGS. 10-11. For example, after the device transmits the second image,validation user interface 222 displays a second image 300 c that isdistinct from image 300 b that was used to train the authenticationmodel 136. In some embodiments, in accordance with a determination thatthe fault review criteria are met for one or more respective validationfaults of a set of validation faults that includes the first validationfault and at least one additional validation fault, informationcorresponding to the one or more respective validation faults istransmitted to the validation system. In some embodiments, in accordancewith a determination that the fault review criteria are met, the devicetransmits (1422) information indicating that a fault is detected in thesecond image. In some embodiments, the information indicating that thefault is detected in the second image is used by validation device 201to display an indication that a fault is detected, highlighting alocation of the fault, indicating a type of the fault, and or requestingconfirmation of the presence, location, and/or type of the fault. Forexample, in FIGS. 10-11, indication 1004 (which indicates that an imagefault has been detected) is displayed. In some embodiments, the devicereceives (1424) information from the validation system (e.g.,information provided by a human reviewer via validation user interface222) confirming the detected fault.

In some embodiments, in accordance with a determination that the faultreview criteria are met, the device determines whether promptingcriteria are met (e.g., based on the validation fault type and/or basedon the validation value). In some embodiments, in accordance with adetermination that prompting criteria are met, the device transmits(1426) information that causes a fault identification prompt (e.g.,prompt 1006) to be displayed by the validation device 201.

In some embodiments, the device receives (1428) information from thevalidation system that corresponds to the first validation fault. Forexample, information corresponding to a location of the first validationfault (e.g., as indicated by selection box 1004, FIG. 11) and/orconfirmation of a suggested validation type is transmitted by validationdevice 201 to authentication server 100 (e.g., via validation server200). In some embodiments, the authentication server stores (e.g., bymachine learning module 126), the information (e.g., received fromvalidation device 201) that corresponds to the first validation fault.

In some embodiments, the device determines (1430) (e.g., using themachine learning module 126) a second validation value that correspondsto a probability that the second image of the second identificationdocument includes a second validation fault, wherein the secondvalidation value is used for determining whether the fault reviewcriteria are met. For example, machine learning module 126 trainsauthentication model 136 to recognize a validation fault as describedwith regard to operation 1406 (e.g., a validation fault other than thepunched document fault, such as a layout irregularity fault) based oninput received via validation user interface 222 (e.g., as describedwith regard to FIG. 9). The machine learning module 126 uses the trainedauthentication model 136 to determine whether the second fault (e.g.,the layout irregularity fault) is present in an image included in thereceived second authentication request.

In some embodiments, the device transmits (1432) information regardingthe second authentication request (e.g., detection of the fault by theauthentication model 136, confirmation of the fault, identification ofthe fault, location of the fault (as identified by authentication model136 and/or by a human reviewer via validation user interface 222) and/orindication of unsuccessful authentication) to a remote device (e.g., theremote device from which the second authentication request was received,such as requesting device 154 or user device 156).

In some embodiments, the method proceeds from operation 1432 tooperation 1402, as indicated at C.

In some embodiments, a quality review comparison is performed in whichone or more results of the determinations made using machine learningmodule 126 (e.g., results including validation information as describedwith regard to operation 1406 and/or a first validation value asdescribed with regard to operation 1412) are compared with results ofcorresponding determinations made by human reviewers. For example, for aset of authentication requests evaluated using authentication model 136,a matching set of authentication requests are provided to humanreviewers (e.g., via a validation user interface 222 as described withregard to FIGS. 5-9). In some embodiments, discrepancies between thedeterminations made using machine learning module 126 and thedeterminations made by the human reviewers are used to train theauthentication model 136. In some embodiments, discrepancies between thedeterminations made using machine learning module 126 and thedeterminations made by the human reviewers are reviewed by a qualitycontrol reviewer. In some embodiments, the quality control reviewresults are used to train the authentication model 136. For example, inaccordance with a quality control review determination that a result ofa review performed by machine learning module 126 differs from a resultperformed by a human reviewer, if the quality control reviewerdetermines that the human review result was the correct result, a weightapplied to one or more parameters of authentication model 136 areadjusted.

It should be understood that the particular order in which theoperations in FIGS. 14A-14C have been described is merely an example andis not intended to indicate that the described order is the only orderin which the operations could be performed. One of ordinary skill in theart would recognize various ways to reorder the operations describedherein.

It will be understood that the flow described with regard to FIGS.14A-14C is an example of flow using an exemplary first transaction andexemplary second transaction. In some embodiments, authentication server100, validation server 200, and/or validation device(s) 201 process manytransactions simultaneously.

FIG. 15 is a flow diagram illustrating a method 1500 for using machinelearning to provide feedback at a device that captures an image for anauthentication request, in accordance with some embodiments. The methodis performed at user device 156, authentication server 100, validationdevice(s) 201, and/or validation server 200. For example, instructionsfor performing the method are stored in the memory 1202 and executed bythe processor(s) 1204 of the user device 156. In some embodiments, partor all of the instructions for performing the method are stored in thememory 102 and executed by the processor(s) 104 of the authenticationserver 100. In FIG. 15, dotted lines are used to indicate optionaloperations.

It should be understood that the particular order in which theoperations in FIG. 15 have been described is merely an example and isnot intended to indicate that the described order is the only order inwhich the operations could be performed. One of ordinary skill in theart would recognize various ways to reorder the operations describedherein.

In some embodiments, the device captures (1502) an image and/or videousing camera 1214. For example, the device captures a reference image300 a of an identification document 302, or the device captures anauthentication image 350 a (e.g., a “selfie” image of the user). In someembodiments, the device captures sound data in addition to image data.For example, the device captures a video recording of the user thatincludes an audio recording.

In some embodiments, the device applies (1504) error correction model1230 to the captured image to determine whether the image includes animage capture error (e.g., fraud indications and/or readability faultsas described with regard to operation 1406).

In some embodiments, the device determines (1506) (e.g., based on theapplication of the error correction model 1230 to the captured image)whether the captured image meets quality requirements. In accordancewith a determination that the captured image does not meet qualityrequirements (e.g., a validation value determined using error correctionmodel 1230 for the captured image is above a threshold uncertaintythreshold), the method proceeds to operation 1508. In accordance with adetermination that the captured image meets quality requirements, themethod proceeds to operation 1516.

In some embodiments, in accordance with a determination that thecaptured image does not meet quality requirements, the device provides(1508) image capture feedback. In some embodiments, image capturefeedback includes an indication of a detected image capture error,instructions for avoiding a detected image capture error, and/orinstructions for improving image quality. In some embodiments, imagecapture feedback is output by output device 1216 (e.g., a display and/ora speaker). For example, image capture feedback is displayed in a camerauser interface 1232 (e.g., as illustrated by feedback message 1304)and/or provided via a text-to-speech module as audio output.

In some embodiments, in accordance with a determination that thecaptured image does not meet quality requirements, the device adjusts(1510) one or image capture parameters of camera 1214. In someembodiments, camera operation module 1224 adjusts one or more parameters(e.g., exposure, white balance, focal length, aperture, sensitivity(ISO), flash setting, and/or capture mode) of camera 1214. In someembodiments, camera operation module 1224 adjusts one or more parametersthat correspond to at least one of one or more errors detected usingerror correction model 1230.

In an illustrative example, the error correction module 1230 determines,based on training data, that a captured image includes excessive glare(e.g., because a portion of a facial image is not visible where facialfeatures would typically be present within a facial image and/or withina reference document; because a portion of text is missing where textwould typically be present within a reference document; and/or because alevel of contrast in a first portion of the document is lower than alevel of contrast in a second portion of the document). In accordancewith a determination that the captured image includes excessive glare,camera operation module 1224 adjusts a property of a flash system (e.g.,by disabling a flash device, reducing flash intensity, activating analternative flash mode, and/or activating a different flash producingdevice).

In some embodiments, in accordance with a determination that thecaptured image does not meet quality requirements, the device uses(1512) training information (e.g., the captured image and/or informationregarding a detected image capture error) to train error correctionmodel 1230. For example, user device 156 stores the training informationor transmits the training information to authentication sever 100.

In some embodiments, in accordance with a determination that thecaptured image meets quality requirements, the device uses (1514)quality information (e.g., the captured image and, in some embodiments,a quality indication that the captured image meets the qualityrequirements) to train error correction model 1230. For example, userdevice 156 stores the quality information and/or transmits the qualityinformation to authentication sever 100.

In some embodiments, the device determines (1516) whether additionalimage(s) are needed. For example, if a previously captured image was areference image 300 and (no authentication image 350 has been captured),the device displays a prompt (e.g., in camera user interface 1232) tocapture the authentication image 350, and if a previously captured imagewas an authentication image 350 and (no reference image 300 has beencaptured or previously provided), the device displays a prompt (e.g., incamera user interface 1232) to capture the reference image 300. Inaccordance with a determination that additional image(s) are needed, themethod proceeds to operation 1502. In accordance with a determinationthat no additional image(s) are needed, the method proceeds to operation1518.

In some embodiments, the device generates (1518) an authenticationrequest 124 using the captured reference image 300 and or authenticationimage 350 (e.g., the images that met the quality requirements).

In some embodiments, the device transmits (1520) the captured referenceimage 300 and or authentication image 350 (e.g., the images that met thequality requirements) to the requesting device 154 (e.g., a device thatgenerates an authentication request 124). In some embodiments, thedevice transmits device information (e.g., a serial number, model,and/or manufacturer of the device) and/or camera information (e.g.,camera type, camera position relative to the device, and/or one or moreimage capture parameters) to the authentication server 100. In someembodiments, the device transmits captured audio and/or video to theauthentication server 100. For example, the captured audio and/or videoare used by device 156 and/or by authentication server 100 to evaluatewhether live capture criteria are met.

FIG. 16 is a flow diagram illustrating a method 1600 for generatingadversarial tests to teach a machine learning system, in accordance withsome embodiments. The method is performed at an authentication server100. For example, instructions for performing the method are stored inthe memory 102 and executed by the processor(s) 104 of theauthentication server computer system 100. In some embodiments, themethod 1700 is performed by the machine learning module 126 (e.g., anadversarial training module 134 of the machine learning module 126).

In some embodiments, the device generates (1602) a set of testauthentication requests. Each test authentication request of the set oftest authentication requests includes a respective test validation fault(e.g., a fault as described with regard to operation 1406).

In some embodiments, the device analyzes (1604) the set of testauthentication requests (e.g., by applying authentication model 136 toone or more images included in the authentication requests) to generatea set of validation values that corresponds to the set of testauthentication requests. For example, machine learning module 126determines validation values for the authentication requests (e.g., asdescribed with regard to operation 1412).

In some embodiments, the device determines (1606) whether a respectivevalidation value of the set of validation values does not meet faultreview criteria (e.g., criteria that are met in accordance with adetermination that the validation value is not within a predefined rangeof a target uncertainty value that corresponds to a test authenticationrequest). In accordance with the determination that the respectivevalidation value of the set of validation values does not meet the faultreview criteria, the device adjusts the machine learning system (e.g.,by adjusting authentication model 136). For example, authenticationmodel 136 includes weights applied to each fault in a set of faults(e.g., as described with regard to operation 1406), and, in accordancewith a determination that the respective validation value of the set ofvalidation values does not meet the fault review criteria, the deviceadjusts a weight applied to the tested validation fault and/or adds anew validation fault.

In an illustrative example, the set of test authentication requestsincludes authentication requests in which a text feature (e.g., fontsize) in a reference image 300 is altered by varying degrees from thesame text feature as found in a legitimate identification document. Ifan invalid text field passes through the machine learning system withouttriggering review, the machine learning system is adjusted to identifythe invalid text.

It should be understood that the particular order in which theoperations in FIG. 16 have been described is merely an example and isnot intended to indicate that the described order is the only order inwhich the operations could be performed. One of ordinary skill in theart would recognize various ways to reorder the operations describedherein.

Features of the present invention can be implemented in, using, or withthe assistance of a computer program product, such as a storage medium(media) or computer readable storage medium (media) having instructionsstored thereon/in which can be used to program a processing system toperform any of the features presented herein. The storage medium (e.g.,the memory 102 and the memory 202) can include, but is not limited to,high-speed random access memory, such as DRAM, SRAM, DDR RAM or otherrandom access solid state memory devices, and may include non-volatilememory, such as one or more magnetic disk storage devices, optical diskstorage devices, flash memory devices, or other non-volatile solid statestorage devices. In some embodiments, the memory 102 and the memory 202include one or more storage devices remotely located from the CPU(s) 104and 204. The memory 102 and the memory 202, or alternatively thenon-volatile memory device(s) within these memories, comprises anon-transitory computer readable storage medium.

Communication systems as referred to herein (e.g., the communicationsystem 108 and the communication system 208) optionally communicate viawired and/or wireless communication connections. Communication systemsoptionally communicate with networks (e.g., the networks 150 and 152),such as the Internet, also referred to as the World Wide Web (WWW), anintranet and/or a wireless network, such as a cellular telephonenetwork, a wireless local area network (LAN) and/or a metropolitan areanetwork (MAN), and other devices by wireless communication. Wirelesscommunication connections optionally use any of a plurality ofcommunications standards, protocols and technologies, including but notlimited to Global System for Mobile Communications (GSM), Enhanced DataGSM Environment (EDGE), high-speed downlink packet access (HSDPA),high-speed uplink packet access (HSUPA), Evolution, Data-Only (EV-DO),HSPA, HSPA+, Dual-Cell HSPA (DC-HSPDA), long term evolution (LTE), nearfield communication (NFC), wideband code division multiple access(W-CDMA), code division multiple access (CDMA), time division multipleaccess (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 102.11a,IEEE 102.11ac, IEEE 102.11ax, IEEE 102.11b, IEEE 102.11g and/or IEEE102.11n), voice over Internet Protocol (VoIP), Wi-MAX, a protocol fore-mail (e.g., Internet message access protocol (IMAP) and/or post officeprotocol (POP)), instant messaging (e.g., extensible messaging andpresence protocol (XMPP), Session Initiation Protocol for InstantMessaging and Presence Leveraging Extensions (SIMPLE), Instant Messagingand Presence Service (IMPS)), and/or Short Message Service (SMS), or anyother suitable communication protocol, including communication protocolsnot yet developed as of the filing date of this document.

It will be understood that, although the terms “first,” “second,” etc.may be used herein to describe various elements, these elements shouldnot be limited by these terms. These terms are only used to distinguishone element from another.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the claims. Asused in the description of the embodiments and the appended claims, thesingular forms “a,” “an” and “the” are intended to include the pluralforms as well, unless the context clearly indicates otherwise. It willalso be understood that the term “and/or” as used herein refers to andencompasses any and all possible combinations of one or more of theassociated listed items. It will be further understood that the terms“comprises” and/or “comprising,” when used in this specification,specify the presence of stated features, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, steps, operations, elements, components, and/orgroups thereof.

As used herein, the term “if” may be construed to mean “when” or “upon”or “in response to determining” or “in accordance with a determination”or “in response to detecting,” that a stated condition precedent istrue, depending on the context. Similarly, the phrase “if it isdetermined [that a stated condition precedent is true]” or “if [a statedcondition precedent is true]” or “when [a stated condition precedent istrue]” may be construed to mean “upon determining” or “in response todetermining” or “in accordance with a determination” or “upon detecting”or “in response to detecting” that the stated condition precedent istrue, depending on the context.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific embodiments. However, theillustrative discussions above are not intended to be exhaustive or tolimit the claims to the precise forms disclosed. Many modifications andvariations are possible in view of the above teachings. The embodimentswere chosen and described in order to best explain principles ofoperation and practical applications, to thereby enable others skilledin the art.

1. (canceled)
 2. A computer-implemented method, comprising: at a serversystem including one or more processors and memory storing one or moreprograms for execution by the one or more processors: receiving, over anetwork from a user device remote from the server system, a firstauthentication request to authenticate an identity of a user, whereinthe first authentication request includes at least a first image thatcorresponds to a first identification document; determining, using amachine learning system configured to detect one or more validationfaults of a plurality of fault types, a confidence value that indicatesa probability that the first image includes a first validation fault ofthe plurality of fault types; determining whether the confidence valueexceeds a threshold uncertainty value; and in accordance with adetermination that the confidence value exceeds the thresholduncertainty value, transmitting, to a validation system, the firstimage.
 3. The computer-implemented method of claim 2, wherein the firstvalidation fault of the plurality of fault types identifies acorresponding fault location.
 4. The computer-implemented method ofclaim 2, wherein: in accordance with a determination that the confidencevalue does not exceed the threshold uncertainty value, transmitting thefirst authentication request to a remote device.
 5. Thecomputer-implemented method of claim 2, wherein: in accordance with adetermination that the confidence value exceeds the thresholduncertainty value: determining whether prompting criteria are met; and,in accordance with a determination that the prompting criteria are met,transmitting information for displaying a prompt for information thatcorresponds to the first validation fault of the plurality of faulttypes in the first identification document.
 6. The computer-implementedmethod of claim 2, including: determining, using the machine learningsystem, a second validation value that corresponds to a probability thatthe first image of the first identification document includes a secondvalidation fault of the plurality of fault types, distinct from thefirst validation fault of the plurality of fault types.
 7. Thecomputer-implemented method of claim 6, including: analyzing, using themachine learning system, a set of incoming authentication requests; anddetermining, by the machine learning system, a subset of the set ofincoming authentication requests that correspond to invalidauthentication request activity, wherein the second validation fault ofthe plurality of fault types is an invalidity characteristic of thesubset of the set of incoming authentication requests.
 8. Thecomputer-implemented method of claim 2, including: generating, using themachine learning system, a set of test authentication requests, whereineach test authentication request of the set of test authenticationrequests includes a respective test validation fault of the plurality offault types; analyzing, by the machine learning system, the set of testauthentication requests to generate a set of validation values thatcorresponds to the set of test authentication requests; determining thata respective validation value of the set of validation values does notmeet fault review criteria; and in accordance with the determinationthat the respective validation value of the set of validation valuesdoes not meet the fault review criteria, adjusting the machine learningsystem using the respective test validation fault of the plurality offault types that corresponds to the respective validation value.
 9. Anon-transitory computer readable storage medium storing one or moreprograms, the one or more programs comprising instructions, which whenexecuted, cause a server system to: receive, over a network from a userdevice remote from the server system, a first authentication request toauthenticate an identity of a user, wherein the first authenticationrequest includes at least a first image that corresponds to a firstidentification document; determine, using a machine learning systemconfigured to detect one or more validation faults of a plurality offault types, a confidence value that indicates a probability that thefirst image includes a first validation fault of the plurality of faulttypes; determine whether the confidence value exceeds a thresholduncertainty value; and in accordance with a determination that theconfidence value exceeds the threshold uncertainty value, transmit, to avalidation system, the first image.
 10. The non-transitory computerreadable storage medium of claim 9, wherein the first validation faultof the plurality of fault types identifies a corresponding faultlocation.
 11. The non-transitory computer readable storage medium ofclaim 9, wherein the one or more programs include instructions thatcause the server system to, in accordance with a determination that theconfidence value does not exceed the threshold uncertainty value,transmit the first authentication request to a remote device.
 12. Thenon-transitory computer readable storage medium of claim 9, wherein theone or more programs include instructions that cause the server systemto, in accordance with a determination that the confidence value exceedsthe threshold uncertainty value: determine whether prompting criteriaare met; and, in accordance with a determination that the promptingcriteria are met, transmitting information for displaying a prompt forinformation that corresponds to the first validation fault of theplurality of fault types in the first identification document.
 13. Thenon-transitory computer readable storage medium of claim 9, wherein theone or more programs include instructions that cause the server systemto: determine, using the machine learning system, a second validationvalue that corresponds to a probability that the first image of thefirst identification document includes a second validation fault of theplurality of fault types, distinct from the first validation fault ofthe plurality of fault types.
 14. The non-transitory computer readablestorage medium of claim 13, wherein the one or more programs includeinstructions that cause the server system to: analyze, using the machinelearning system, a set of incoming authentication requests; anddetermine, by the machine learning system, a subset of the set ofincoming authentication requests that correspond to invalidauthentication request activity, wherein the second validation fault ofthe plurality of fault types is an invalidity characteristic of thesubset of the set of incoming authentication requests.
 15. Thenon-transitory computer readable storage medium of claim 9, wherein theone or more programs include instructions that cause the server systemto: generate, using the machine learning system, a set of testauthentication requests, wherein each test authentication request of theset of test authentication requests includes a respective testvalidation fault of the plurality of fault types; analyze, by themachine learning system, the set of test authentication requests togenerate a set of validation values that corresponds to the set of testauthentication requests; determine that a respective validation value ofthe set of validation values does not meet fault review criteria; and inaccordance with the determination that the respective validation valueof the set of validation values does not meet the fault review criteria,adjusting the machine learning system using the respective testvalidation fault of the plurality of fault types that corresponds to therespective validation value.
 16. A server system, comprising: one ormore processors; memory; and one or more programs, wherein the one ormore programs are stored in the memory and are configured for executionby the one or more processors, the one or more programs includinginstructions for: receiving, over a network from a user device remotefrom the server system, a first authentication request to authenticatean identity of a user, wherein the first authentication request includesat least a first image that corresponds to a first identificationdocument; determining, using a machine learning system configured todetect one or more validation faults of a plurality of fault types, aconfidence value that indicates a probability that the first imageincludes a first validation fault of the plurality of fault types;determining whether the confidence value exceeds a threshold uncertaintyvalue; and in accordance with a determination that the confidence valueexceeds the threshold uncertainty value, transmitting, to a validationsystem, the first image.
 17. The server system of claim 16, wherein thefirst validation fault of the plurality of fault types identifies acorresponding fault location
 18. The server system of claim 16, whereinthe one or more programs include instructions for, in accordance with adetermination that the confidence value does not exceed the thresholduncertainty value, transmitting the first authentication request to aremote device.
 19. The server system of claim 16, wherein the one ormore programs including instructions for, in accordance with adetermination that the confidence value exceeds the thresholduncertainty value: determining whether prompting criteria are met; and,in accordance with a determination that the prompting criteria are met,transmitting information for displaying a prompt for information thatcorresponds to the first validation fault of the plurality of faulttypes in the first identification document.
 20. The server system ofclaim 16, wherein the one or more programs including instructions for:determining, using the machine learning system, a second validationvalue that corresponds to a probability that the first image of thefirst identification document includes a second validation fault of theplurality of fault types, distinct from the first validation fault ofthe plurality of fault types.
 21. The server system of claim 20, whereinthe one or more programs including instructions for: analyzing, usingthe machine learning system, a set of incoming authentication requests;and determining, by the machine learning system, a subset of the set ofincoming authentication requests that correspond to invalidauthentication request activity, wherein the second validation fault ofthe plurality of fault types is an invalidity characteristic of thesubset of the set of incoming authentication requests.